DOCUMENT RESUME 

ED 069 743 TM 0C2 198 

Evaluation of Innovative Schools: Student 
Achievement, 1970-71. 

Broward County School Board, Fort Lauderdale, Fla. 

Apr 72 

24p.; Research Dept. Report No. 55 
MF-$0.65 HC-$3.29 

♦Academic Achievement; Comparative Analysis; 
Conventional Instruction; *Educational Innovation; 
Elementary Grades; ^Evaluation; *0pen Plan Schools; 
Primary Education; Racial Factors; School Design; Sex 
Differences; student Ability; Traditional Schools 
Broward County Public Schools; Florida 

ABSTRACT 

Third-, fifth-, and eighth-grade pupil achievement 
test results in the 1970-71 school year are contrasted in terms of 
types of school plants, with those schools that had b^en in operation 
in a recently constructed school plant for at least ope vear grouped 
as "Innovative." Differences in ability levels of pupils‘were taken 
into account in all the reported analyses. Pupils in innovative 
plants made their strongest showing at the third-grade level, with 
boys stronger than girls. White third— grade boys* test results 
favored conventional -schools, black boys, innovative schools. .At the 
fifth-grade level white boys' test scores again favored conventional 
plants; eighth- year test results generally favored conventional 
plants for all sex/race groups except black boys. Correlation is 
drawn between the test results and the length of time the fifth- year 
pupils had attended school in the county school system. Causal 
interpretations of the findings are dependent upon further analysis 
of the data and on future longitudinal studies to reduce the 
speculative elements involved' in accepting data-based hypotheses 
about the programs' effectiveness. (Author/LH) 
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ABSTRACT 



This report contrasts 1970-71 third-, fifth-, and eighth-grade pupil 
achievement test results in terms of types of school plants. Schools 
grouped as "Innovative" were those that had been in operation in a 
recently constructed school plant for at least one year. Differences 
in ability levels of pupils were .taken into account in all the reported 
analyses . 

Pupils in innovative plants made their strongest showing at the third- 
grade level. At this grade trends involving boys were stronger than 
those involving girls. White third-grade boys' test results favored 
conventional schools, black boys' innovative schools. 

At the fifth-year level white boys' test scores again favored conven- 
tional plants. Except for white girls' scores on the Math Computation 
test, no other particularly salient trends were evident. White fifth- 
year pupils of both sexes in conventional plants showed a strong 
tendency to score higher on the Math Computation test. 

Eighth-year test results generally favored conventional plants for all 
sex/race groups except black boys. The Math Concepts test was the 
largest single contributor to this difference for whites of both sexes. 
The Language Mechanics test was the best discriminator between school 
types for black girls. 

An interesting finding regarding the length of time fifth-year pupils 
have attended schools in Broward County is briefly mentioned in this 
report. White fifth-year pupils of both .sexes who had been in local 
public schools one or two years scored about the same in both types of 
schools on the Math Computation test. Those with three or more years 
in Broward scored higher on this test in conventional schools. The 
Math Computation test is Important because it was the most important 
source of differences between school types for white fifth-year pupils. 

It would at present be very speculative to attribute the findings of 
this report to such factors as overcrowding or to problems inherent in 
the new physical plants. Many factors Influence test results. It is 
always difficult to be confident about causal interpretations of find- 
ings in educational research. Future analyses of these data, and 
longitudinal studies will be required in order to reduce the speculative 
elements involved in accepting data-based hypotheses about the effective- 
ness of local innovative programs. 
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INTRODUCE ' J 



This report is concerned with pupil achievement in open-space school 
plants during the 1970-71 school year. Reports of this type are based 
upon two sources of information: 

1. Countywide test results: The tests used were the California 

lest £f Basic Skills ( CTBS.) and the California Test of Mental 
Maturity (CM). These tests were administered to local 
third-, fifth-, and eighth-grade pupils in May, 1971. 

2. Research Questionnaires: These questionnaires were constructed 

cooperatively with instructional personnel. All teachers in 
innovative schools were given an opportunity to participate in 
developing the instruments. The questionnaires were designed 
to provide information about instructional programs and the 
attitudes of pupils and teachers. These instruments were 
administered to a sample of fifth-year teachers and pupils 

in May , 1971. A shorter battery of questionnaires was also 
administered to a sample of eighth-year pupils and teachers. 



PURPOSE 

This report's primary purpose is to contrast achievement, after allowing 
for differences in ability, in open-space and conventional school plants.* 
All pupils who took the tests are included in the primary analyses. The 
sampling studies and research questionnaire, information is only briefly 
alluded to in this report. 

This report is important because it provides a basic summary of achieve- 
ment data by types of school plants. Any subsequent reports must seek 
to clarify or explain this basic information. At the end of this report 
an example of such a "clarification" or "explanation" is briefly cited. 



ft PROCEDURES 

At the eighth-year level the following schools were classified as 

Innovative": Apollo, Lauderdale Lakes, Lauderhill, Plantation, Rickards, 

and Nova. ’ 

At the third- and fifth-year level the following schools were classified 
as Innovative : Palmview, Coconut Creek, Cypress, Floranada, Castle Hill 

Sabal Palm, Village, Mirror Lake, Hollywood Park, Hollywood Central, 
Annabel C. Perry, Fairway , and the two Novas. 

\ This division was primarily based upon type of school plant. Schools 

were onl y included if they had been operating in open-space plants during 
\ roost of the 1970-71 school year.* 

i 

Preliminary analyses of the data indicated sex and race differences 
I within grades which introduced great' complexities in attempting to expli- 

| cate relationships between IQ scores, sex, race, and type of school. For 
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o^jS e i 0f , Si !‘ plici ^» each grade ’ sex ’ and race group was then analyzed 
separately in terms of ability differences and type of school. The 

latter analyses form the primary basis for the present report. 

procedaras are further amplified and some documentation of the 
results is provided in the appendix to this report. 

RESULTS 

The following tables use a simplified format to convey significant dif- 
ferences in a non-numeric form. Results are presented for each of ten 
sub tests. Sub tests are used because they provide the maximum amount of 
diagnostic information. 

Results are coded as follows: 

1. Type of School: C = Conventional 

I ■ Innovative 



2. Ability Level: 



If a box is coded "C" 



H 
A 
L 

II X * * 



High (IQ over 116) 

Average (IQs between about 84-116) 
Low (IQ below 84) 



or "I" it means that students at all levels of 



ability in that particular type of school tended to score significantly 
higher on that suhtest. If an "L/ "A/," or "II/" precedes the code, it 
means that differences mainly favored the designated ability group(s) 
d-or that type of school. For example, "L/C" would mean low-ability pupils 
scored higher in conventional schools. Average and high-ability pupils 
did not differ to any noteworthy extent. 

Differences marked with an represent cases where differences are 
quite salient. Such variables represent the most important sources of 
differences between the types of schools. Empty boxes signify the 
absence of significant differences. 

In interpreting the tables the reader its advised that the starred dif- 
ferences are most worthy of confidence. Less confidence should be 
placed in the single letters which bear no stars. In general, the least 

confidence should be placed in the differences preceded by ability-level 
prefixes. * 



Third Year : 

° f 4 T u ble J indicates that the Math Computation test served to 
best distinguish achievement for third-year white boys, favoring conven- 

plan Jf* 0thar significant differences, with the exception of high- 
ability pupils on the Math Concepts subtest, also tended to fav 0 r con- 
ventional schools. 

White girl s did not differ significantly on any of the subtests. It 
should he noted, however, that white girls did differ significantly when 
all variables were tested simultaneously (multivariate test of signifi- 
cance). The main- sources of this multivariate pattern were the Math 
Co mputation test (in favor of conventional schools) and the Langllage 
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Table I 



Third-Year Results 



1 — ■ ■ - ■ ' 
Tests 


White 

Boys 


White 

Girls 


Black 

Boys 


Black 

Girls 


Reading Vocabulary 


L/C 








Reading Comnrehension 


C 








Language Mechanics 






I* 




Language Expression 






I 




Language Spelling 


L/C 








Math Computation 


C* 




I* 


I 


Math Concepts 


L/C, H/I 




I 




Math Application 






L/I 


I 


Study Skills Total 






I 





Mechanics and Spelling tests .(in favor of innovative schools) . However, 
as mentioned previously, no single test by itself differentiated signifi- 
cantly and - the overall (multivariate) pattern was not at a highly 
significant level (P ^ .047). It is, therefore, inappropriate to discuss 
the direction of an overall difference since the subtests point in 
different directions. The majority of the tests, however, favored inno- 
vative-schools, even though these differences failed to attain statistical 
significance. 

Significant differences among black boys for the most part favored inno- 
vative schools. The Language Mechanics and Math Computation tests were 
highly significant sources of differences. 

The two significant differences among black girls also favored innovative 
schools. In this case, however, no single variable was highly significant. 

Overall trends tended in the direction of favoring innovative schools for 
blacks and conventional schools for whites at the third year. In general, 
these trends tend to exist largely for boys of both races and are not 
particularly evident among girls. 

Fifth Year ; 

Results for white fifth-grade boys were relatively unambiguous. Only the. 
Math Computation test is starred, but differences in Language Mechanics , 
Spelling , and Math Concepts were also fairly large. The reason that the 
latter two subtests are not starred is that the Math Computation test 
•clearly stood out as the largest single contributor to the "factor" which 
these tests have in common and in terms of which the two types of schools 
differed the most. In brief, the aforementioned tests were somewhat 
redundant contributors to the "factor" which maximally discriminated 
between the two types of schools. All of the subtests contributed in the 
same direction to this "factor" (all favored conventional schools). 

Results for white girls were not as clear cut. The Math Computation test 
was again, by far, the largest single contributor to differences among 
white girls in the two types of plants and was consistent with results 
for white boys. It may be confidently concluded that this subtest did a 
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Table II 



Fifth-Year Results 



Tests 


White 

Boys 


White 

Girls 


Black 

Boys 


Black 

Girls 


Reading Vocabulary 










Reading Comprehension 










Language Mechanics 


C 








Language Expression 




L/C, H/I 




C 


Language Spelling 


C 








Math Computation 


C* 


C* 






Math Concepts 


C 


C 


L/I ! 




Math Applications 


C 




1 

f 


Study Skills Reference 


C 




C ! 


Study Skills Graphics 






1 



good job of detecting differences among white fifth-year pupils of both 
sexes in terms of the two types of schools. Unlike the results for 
white boys, however, not all tests favored the conventional schools. 

The two reading tests in particular showed a counter trend. Since the 
only significant differences favored conventional schools, the reading 
test results should not be emphasized. They are noted because they serve 
to indicate a sex difference between overall trends for white fifth-year 
pupils. 

Other than. for the differences noted in the table, no clear-cut trends 
seemed apparent from the analyses involving black boys. The Language 
Expression test, while not significant, was the second largest contribu- 
tor to differences for black boys among the types of schools. This is 
noted because this tist was significant in the same direction (favoring 
conventional schools) for black girls. 

Black girls did not differ significantly on the Study Skills Reference 
test. This test was, however, a relatively strong secondary contributor 
to the "factor" which best differentiated the performance of black girls 
in the two types of schools. The black girls' study skills scores were 
similar to the black boys' results on this test. Blacks of both sexes, 
therefore, tended to score higher in conventional schools on the Language 
Expression and Study Skills Reference tests. 

Overall trends favoring conventional schools were stronger for whites 
than blacks at the fifth-year level. The group which clearly differed 
the most at the fifth-year level was the white boys. 

Eighth Year : 

All significant differences involving eighth-year white boys favored 
conventional schools. Scores on the Reading Vocabulary , Language 
Mechanics , and Math Computation tests, while not significant, tended to 
be similar to the findings for other race/sex groups. In other words, 
they made a moderate secondary contribution (in favor of conventional 
schools) to differentiating between the types of schools. These tests 
are mentioned because significant differences were obtained on them for 
one or more of the other race/sex groups. 



Table III 



Eighth-Year Results 



Tests 


White 

Boys 


Whi te 
Girls 


Black 

Boys 


Black 

Girls 


Reading Vocabulary 




C 






Reading Comprehension 




L/C, A/ I 




C 


Language Mechanics 




L/C 


C 


C* 


Language Expression 


C 




H/I 




Language Spelling 


C 


C 






Math Computation 


i 


C 






Math Concepts i C* 


C* 






Math Applications C 


C 


• A/C 


H/C 


Study Skills Reference i 








Study Skills Graphics j 


L/I, Ml 







The general direction of white girls' scores, with the exceptions of the 
Study Skills and Reading Comprehension tests, also favored conventional 
schools. Language Expression was the only test which did not show a 
fairly strong direction one way or another. The Math Concepts test was 
the best single differentiator of school types for both white boys and 
girls. 

Black boys differed as noted on the chart for individual tests. The 
overall difference, considering all tests simultaneously (multivariate 
test), however, did not reach statistical significance at the .05 level 
(obtained P-£-.076) ( . On this basis, it may be concluded that the pattern 
of achievement did not differ by very much in terms of the two types of 
school plants. •" 

Black girls in the two types of schools differed considerably on the 
Language Mechanics cyst. Most test scores for this group (except 
Reading Vocabulary and Study Skills Reference) also show at least a 
moderately strong tre»y4 in a direction favorable to conventional schools. 
It can be concluded that eighth-grade test results generally favored 
conventional plants for all sex/race groups with the exception of black 
boys. 

DISCUSSION AND CONCLUSIONS 

These results can be best interpreted by instructional personnel. 

Only a few comments will be made about the findings in this section. 

It would appear that the innovative schools make their strongest showing 
at the third-year level, especially among black pupils. It is at this 
grade level that pupils have had less experience with conventional 
plants and programs. Personnel in innovative schools may be somewhat 
encouraged by these results. 

At- the elementary level important differences on subtests tended to 
be most pronounced on skill subjects which seem to lend themselves to 
drill and rote procedures. This may reflect a difference in program 
emphasis, but it may also be that these are the only tests in the battery 
which are very sensitive to any differences at all in school programs. 



These results cannot be taken as providing definitive evidence about 
the superiority of one type of program to another.. It is a big jump 
to go from test results to conclusions about schools or programs. 

Many factors other than the quality of pn instructional program can 
influence test results. 

Factors which are apparently connected with the quality of an instruc- 
tional program, such as staffing and overcrowding, are often cited as 
problems by personnel in innovative schools. For this reason, they may 
feel that evaluations such as this one present an unfair picture. 
Questionnaire results indicate that a majority of the teachers in the 
new school plants take a favorable position toward the innovative ideas 
which are being tested in their schools. It would appear that they are 
somewhat unhappy with the implementation of innovative ideas, but not 
with the ideas themselves. They do not want to see these ideas 
threatened by unfavorable evaluative results. ' 

It seems premature to this writer to jump to the conclusion that results 
contained in this report are largely due to differences in school pro- 
grams or plants. Causality is a difficult issue in educational research. 
However, instructional personnel may be correct in thinking that innovative 
programs have been hurt by overcrowding and understaffing. It is as 
reasonable to attribute unfavorable results to these factors as it is to 
attribute them to individualized instruction or open-space plants. 

No conclusions will be drawn in this report. The next section, however, 
will allude to current and projected investigations which may contribute • 
to a further understanding of these results. 



ONGOING STUDIES INVOLVING STUDENT ACHIEVEMENT 

Research data collection during the 1970-71 school year was designed to 
provide source material for testing a variety of exploratory hypotheses 
concerning not only innovative schools, but the overall instructional 
program of the county. 

From a methodological standpoint, some of the results of these explora- 
tory studies cannot be viewed with a great deal of confidence until 
they can be validated in subsequent studies. Many analyses of the data 
collected in 1970-71 have already been carried out. Some of the more 
salient findings have been, or will be, communicated informally to 
instructional personnel. , 

The most potentially useful research currently under way or projected, 
will examine complex relationships involving teacher and program 
variables, perceptions and attitudes, achievement and ability. It is 
likely that some of the findings of such studies will not hold up over 
time. Sometimes it will be because the findings have no real basis in 
fact and only reflect a chance permutation of variables. In other cases, 
the finding may have been valid but the circumstances could have altered 
in the interim between evaluations. 

A finding, which may be of some interest but which must be regarded 
as highly tentative, will be briefly outlined in this section. This 
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finding concerns fifth-year pupils because it was at this level that 
most of the supplemental research data were collected. 

The finding involved white fifth-year pupils of both sexes. The number 
of years^such pupils had attended school in Broward County seemed to 
explain differences between types of schools on the Math Computation 
test. It will be recalled that this test was the one on which white 
fifth-year pupils in innovative and conventional schools differed the 
most.^ White pupils who had attended public schools in Broward County 
for three or more years scored higher on this test in conventional 
schools. Pupils who had attended local public schools one or two years 
scored about the same in both types of schools. 



One could be more confident about the above finding had it held for 
more than one test. Analyses designed to uncover the above types of 
trends are continuing. If enough material that can be regarded with 
confidence emerges, it will be documented and formally reported. 



An important set of analyses will commence in the near future. These 
analyses will center upon instructional programs and teacher variables. 
For example, only about half of the fifth-year teachers in the county 
during the 1970-71 school year indicated that they taught in a "self- 
contained situation. It would appear that contrasting "self-contained" 
classes with innovative classes is superior to making contrasts in terms 
ox types of school plants. Another example of an exploratory hypothesis 
to be tested concerns examining whether the colleges teachers attended 
had any effect upon student performance. Other teacher variables to be 
examined on their own and for their possible effects upon differences 
between innovative and conventional programs include sex, years of 
experience, and types and number of degrees. w 



The list of planned and potential long- and short-range evaluative effort* 
made possible through research data collection and improved data proces- 
sing operations is long. It is highly relevant to the interpretation of 
this report, however, to keep in mind that: 



1. evaluative efforts are only entering their initial stages. 

2. the best and most reliable evaluative information can often 
only be confidently established through more than one study. 

3. programs and schools change over time. It required several 
years for the Nova schools to overcome "growing pains" and 
establish a program which, for the most part, tends to yield 
favorable evaluative results. Good longitudinal research may 
help speed up this process for local schools in the future. 
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APPENDIX 



ATPENDIX 



This report will be read by persons of widely varying degrees of statis- 
tical competence. Copies of printouts giviug very si.:. pie summary 
information about numbers of: cases, means, and standard deviations are 
provided in this section. We have also reproduced copies of the most 
pertinent pages of the statistical output associated with this study. 

These pages contain the tests of significance, contrasts, standard 
errors, etc. These copies will be made available upon request to anyone 
Interested in obtaining them. 

The remainder of this section will be devoted to briefly outlining the 
technical procedures followed ir.: summarizing the data. All data analyses 
were carried out upon raw scores on the achievement subtests. Obtained 
IQ scores were used as covariates. Subtests were used, rather than total 
scores, to permit the application of multivariate procedures. However, 
total scores were also processed in each analysis. 

The analyses which were most fundamental to this report utilized 
Eliot Cramer's version of HANOVA* (multivariate analysis of variance). 

The printouts which have -been copied for dissemination to readers , 
interested in examining technical data all derive from this source. 

Tests were also made for curvilinear relationships involving the IQ 
covariate. The program used to do this was a modified version of the 
multiple regression program provided by IBM through its scientific sub- • 
routine package. This program was modified by the Research Department 
to yield regression analysis output similar to that popularized by Ward.** 
This program was also used to -predict scores at selected IQ levels so that 
trends could be examined in cases where tests for equality of regression 
were significant. Inspection of these trends formed the basis for pre- 
fixing ability-level differences in reporting results for comparisons 
involving significantly different regression coefficients. 

It should be noted that all variables were analyzed in MANOVA with and 
without a squared cov&fiate. Copies of results for both models have been 
reproduced. In most cases, inclusion of a squared term did not materially 
influence results. Differences favoring third-grade black girls in ' 
innovative schools were,, however, considerably reduced by introducing a 
curvilinear model. 

The .05 level of significance was used as a basis for making entries in 
the non-numeric summary tables. Where differences existed between the 



* Cramer, Eliot and Sherin, Richard J. MANOVA, Multivariate Analysis of 
Variance . A program distributed by Clyde Computing Service, Box 166~ 
Coconut Grove Station, Miami, Florida. 

** Ward, J. 11., Jr., "Multiple Linear Regression Models," in 11. Borko, ed., 
Computer Applications in the Behav ioral Sciences, Englewood Cliffs. N.J. 
Prentice-Hall, 1962. 



linear and curvilinear models, the latter were given priority. Inter- 
actions between school types and IQ scores may in some instances be 
more accurately portrayed by allowing for one inflection in regression 
lines. 

The discussions in the report skip about somewhat between univariate 
and multivariate interpretations. Since the number of cases in each 
comparison was rather large, it may well be that only the univariate 
differences which were starred in the report are really of much practical 
importance. These differences usually represented over twenty percent of 
the standard error for a variable, were significant at or beyond the .001 
level, and loaded at least in the fifties in terms of their correlation 
with the multivariate discriminant function score. Discussions based 
upon the multivariate results were motivated by the need to indicate 
those instances in which most subtests showed no particularly strong 
tendency to favor one school type. 

In preparing the data for analysis, a missing data multiple regression 
program was used to estimaC': missing scores for a small proportion of 
students. This was done to permit multivariate analyses of the data. 
Scores were estimated separately by sex, grade, and race groups. Scores 
were only estimated for pupils who were missing less than half of a 
possible eleven subtests and IQ variables. The proportion of students 
with one or more estimated score did not exceed five percent for any 
sex/race/grade group. Results for all students who took each test were 
compared with results for the groups used in' this report. Differences 
were found to be trivial. 
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